Running tests to verify the performance of software systems is important for maintaining the integrity of the software systems. Often events such as installation of new software modules, updating, and/or customizing software modules, require running many tests in order to verify that after the software systems still operate as expected. Since such software systems are often large and complex, e.g., SAP ERP or Oracle EBS may include many software modules, testing can take up a large portion of the information technology (IT) budgets of the organizations.
Since many organizations utilize same or similar software modules (e.g., modules belonging to standard software packages), they often end up running similar tests. Collecting testing data from multiple organizations may enable those and/or other organizations, to utilize each other's testing-related knowledge, which is in a sense a wisdom of the crowd (of testers). Consequently, they might be able to come up with a more effective and efficient testing plan. That being said, given the diversity in software systems and customizations, and the large and diverse body of tests performed to validate the software systems, it may not be an easy task to select tests that are likely to be beneficial. While some tests that are run may be effective and test relevant aspects of systems, others may be ineffective, erroneous, and/or test esoteric organization-specific aspects of software systems. Thus, though there may be many thousands of transactions that can be tested by a certain organization, it may be difficult to select which transaction to test, and/or which tests are most relevant. Given a desire to select and/or generate a test for a new organization, which of the many tests in the collection of testing data should be considered? An improper selection of the tests may lead to suboptimal results, such as providing the new organization with an irrelevant or ineffective test.